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Abstract 

Managing quality processes become critically important for higher education institutions 
generally, but especially for institutions involved in open and distance learning. In Australia, 
managers of centers responsible for open and distance learning have identified two frameworks 
that potentially offer ways of conceiving of the application of quality processes: the Quality 
Framework published in Inglis, Ling, and Joosten (1999); and the Benchmarking Framework 
published in McKinnon, Walker, and Davis (2000). However, managers who have been 
considering applying one or other framework within their institutional contexts have had to face 
the issue of how they should choose between, or combine the use, of these frameworks. Part of 
their dilemma lies in distinguishing among the related functions of quality improvement, quality 
assurance, and benchmarking. This article compares the frameworks in terms of their scope, 
institutional application, structures, and method of application, and then considers what 
implications the similarities and differences between the frameworks have for their use. 

Introduction 

Quality has become a matter of major importance for higher education institutions generally, but 
particularly so for institutions involved in open and distance learning (ODL) (Higher Education 
Council, 1997; Twigg, 2000; Western Cooperative for Educational Telecommucations, 2003). 
The origins of the quality movement within distance education in Australia can be traced back at 
least to the study by Nunan and Calvert (1992) and even before that to the work of the Standing 
Committee on External Studies in the mid 1980s. The Open and Distance Learning Association 
of Australia chose quality as the theme of its biennial forum in 1991 (Atkinson, McBeath and 
Meacham, 1991). At about the same time, the Australian government, through the National Board 
of Employment, Education, and Training, funded a study of costs and quality in on- and off- 
campus resource-based learning (Jevons and Northcott, 1994). 

Within the broader higher education sector, the interest in quality originated from previously 
established movements overseas as it did from the debate taking place within distance education. 
The establishment of the Australian Universities Quality Agency (AUQA) to manage institutional 
quality audits within universities was influenced mainly by developments in the United Kingdom, 
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mainland Europe, and the United States, and by the prior establishment of a similar agency in 
New Zealand (Vidovich, 2002). 

The issue of quality impinges on the work of universities in a number of ways. For example, in 
order to protect their critical market (see also Cummings, Phillips, Tilbrook and Lowe; Nunan; 
Reid; Smith; McConachie, Danaher, Jones and Luck, this issue) in overseas education, 
universities have wanted to ensure that the standard of the educational products that they have 
been offering matches the standard of what they are offering onshore. Universities are also keen 
to establish how they compare with one other, or at least with other similar universities, even if 
they are not always keen to make such comparisons public. 

The growing concern with quality in higher education has led institutions to look for ways of 
managing quality processes. In recent times, the focus of attention has turned to quality processes 
in the context of the online delivery of programs. This in turn has led to various attempts to 
develop frameworks for conceptualizing and structuring these processes. In Australia, the Quality 
Improvement Framework described in Inglis, Ling, and Joosten (1999) and the Benchmarking 
Framework described in McKinnon, Walker, and Davis (2000) have offered alternative ways of 
supporting quality processes in relation to teaching and learning in higher education. Staff in a 
number of universities have been looking into how each of these frameworks may be used to 
manage quality processes within distance, open, and flexible learning, including online learning. 
As a result, the author has received a number of enquiries seeking advice on how the two 
frameworks should be viewed in relation to each other. From the fact that these enquiries were 
being received, it seemed that comparison of the frameworks might be useful at this point of time. 
Furthermore, it was felt that if such a comparison were to be attempted by one of those involved 
in the development of one or other of the frameworks it might be regarded as all the more useful. 

This article therefore attempts to compare the two frameworks from the point of view of the 
practitioner in ODL who wants to make a choice between the two. It couches this comparison in 
terms of four factors: the scope of the frameworks; the type(s) of institution to which they are 
meant to be applied; the structures of the frameworks; and the ways in which the frameworks are 
intended to be used. Based on this comparison, it then considers the implications of the 
similarities and differences between the frameworks for their use in managing quality processes 
in ODL. 


Distinguishing Among Quality Processes 

In the literature related to quality in higher education, three terms commonly appear: 
benchmarking, quality assurance, and quality improvement. These terms refer to distinct yet 
closely related functions. Judgments about quality are comparative and what distinguish among 
these three functions are the types of comparisons that are made. 


Benchmarking 

‘Benchmarking’ (see also Reid, this issue) is a term that is now widely used within the quality 
arena. Benchmarking involves comparing a set of products or services against the best that can be 
found within the relevant industry sector. 

The European Benchmarking Code of Conduct defines benchmarking as being “the process of 
identifying and learning from Good Practices in other organizations” (The European Federation 
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of Quality Management, n.d., p. 1). The Public Sector Benchmarking Service in the United 
Kingdom describes benchmarking as involving: 

Regularly comparing aspects of performance (functions or processes) with best 
practitioners, identifying gaps in performance, seeking fresh approaches to bring 
about improvements in performance, following through with implementing 
improvements, and following up by monitoring progress and reviewing the 
benefits. (Public Sector Benchmarking Service, n.p.) 

O’Reagain and Keegan (2000) have described the four steps involved in benchmarking as: 1) 
understanding in detail one’s own processes; 2) analyzing the processes of others; 3) comparing 
your own performance with that of others analyzed; and 4) implementing the steps needed to 
close the performance gap. However, as often happens when a term comes into everyday use, 
some of the original precision of its meaning is lost. Within the higher education sector in 
Australia, the term ‘benchmarking’ is now sometimes used to refer to processes that are more 
concerned with the other quality functions. 

In the area of distance education, various sets of guidelines have been produced to support good 
practice (Twigg, 2001). Having reviewed the literature on guidelines, the Institute for Higher 
Education Policy produced a set of 24 benchmarks by which success in the online delivery of 
programs could be judged (Phipps and Merisotis, 2000). 

Quality Assurance 

Quality Assurance (see also Inglis and McConachie, Danaher, Luck and Jones, this issue) is a 
process oriented to guaranteeing that the quality of a product or a service meets some 
predetermined standard. Quality assurance makes no assumptions about the quality of competing 
products or services. In practice, however, quality assurance standards would be expected to 
reflect norms for the relevant industry. The process of quality assurance therefore compares the 
quality of a product or service with a minimum standard set either by the producer or provider or 
by some external government or industry standards authority. By rights, this standard should bear 
some relationship to best practice, but this is not a necessary condition. The aim in quality 
assurance is to ensure that a product or service is fit for the market. 

Quality Improvement 

Quality Improvement (see also Cummings, Phillips, Tilbrook and Lowe, this issue) is concerned 
with raising the quality of a product or service. The type of comparison that is made when 
engaged in quality improvement is between the current standard of a product or service and the 
standard being aimed for. Quality improvement is concerned with comparing the quality of what 
is about to be produced with the quality of what has been produced in the past. Quality 
improvement is therefore primarily concerned with self rather than with others. Processes focused 
on quality improvement are also focused more on specific aspects of an organizational unit’s 
performance than on overall performance. It is usually the case that constraints dictate that efforts 
at improvement need to be targeted at areas of greatest need. 

Quality in Open and Distance Learning 

ODL differs in numerous respects from the practices employed in face-to-face education and the 
practices employed in online ODL differ from those involved in more traditional forms of ODL. 
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The ways in which quality comparisons are made need not only to recognize that these 
differences exist, hut also to take them into account when the processes for judging quality are 
designed. It is not appropriate to judge the quality of programs offered online hy the same criteria 
as those used to judge the quality of programs offered face-to-face or hy print-hased distance 
education, even though in overall terms some comparahility among differing offerings of the 
same program in different modes may he sought. By the same token, however, it is important to 
recognize that there is a high degree of commonality. The overall design of a course and its 
assessment are likely to he unaffected hy the mode, and many of the same resource materials will 
he used in different modes. Where differences are likely to he most pronounced will he in the area 
of delivery of the program and in student support. 

What Is Meant by a “Framework?” 

What are being compared in this paper are what have been described as two “frameworks.” Each 
framework also has associated instructions and examples, and both frameworks are each 
embedded in larger documents. The frameworks themselves, however, are the focus of interest 
here. It is therefore useful to begin by clarifying what is meant in this context by a framework. 

A framework is generally something that provides form and a degree of rigidity. The framework 
of a building gives the building its structural integrity. The types of frameworks being described 
here are intended to bring structure to processes to which they are applied. They are intended to 
assist institutions to organize the processes that are being used in the area of quality management. 
They don’t completely prescribe quality processes, but they channel processes in particular 
directions that accord with best practice. 

While it is possible for both of the frameworks compared to be used for quality improvement, 
quality assurance, and benchmarking, for the purposes of clarity the framework described in 
McKinnon, Walker, and Davis (2000) will, from here on, be referred to as the “Benchmarking 
Framework,” while the framework described in Inglis, Ling, and Joosten (2002) will be referred 
to as the “Quality Improvement Framework.” Benchmarking and quality improvement are the 
processes to which the respective frameworks are most suited. This does not mean, however, that 
they cannot be applied to the other quality processes. This issue is discussed in more detail below. 

Origins of the Frameworks 

The Benchmarking and the Quality Improvement Frameworks were both the products of 
government-funded projects. Although developed within a year of each other, they were produced 
completely independently. The ways in which each of the projects arose differed considerably, 
and the nature of the differences had a major impact on what was produced in each case. 


Origins of the Benchmarking Framework 

The Benchmarking Framework was developed to provide a tool that could be used by universities 
to: 


• Ascertain performance trends and initiate self-improvement 

• Enable groups of universities to compare their performance 
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• Enable universities to ascertain their competitive [positions] (McKinnon, Walker, and 
Davis, p. 1) 

Work on the development of the Benchmarking Framework was undertaken over 18 months in 
1998 and 1999, and the Framework was published in February 2000. The development of the 
Framework involved senior executive staff in 33 Australian universities at the level of vice- 
chancellor, deputy vice-chancellor, pro-vice Chancellor, or deputy principal/ registrar. During the 
Framework's development, discussions were held with staff of the Commonwealth Higher 
Education Management Service. The Framework was published in Benchmarking: A Manual for 
Australian Universities (McKinnon, Walker and Davis, 2000), which can be downloaded from 
the Department of Education, Science, and Training (DEST) website 
(http://www.dest.gov.au/archive/highered/otherpub/Execsumbench.htm) . This document will be 
referred to as "the Manual." 


Origins of the Quality Improvement Framework 

The Quality Improvement Framework originated from a project that was undertaken in order to 
provide expert advice to the National Board of Employment, Education, and Training on the 
issues of quality, cost, and access in resource-based learning by taking advantage of he new 
learning technologies. The project was commissioned on behalf of the Higher Education Council. 
The project had a number of goals, one of which was the development of a framework for 
considering issues of quality in resource-based learning, including issues associated with equity 
of access. 

Following the project’s completion, the main findings together with the Framework were 
published by the Higher Education Council in a booklet entitled Quality in Resource Based 
Learning (Higher Education Council, 1997). However, a more extensive explanation of the 
Framework and its use was subsequently provided in the book Delivering Digitally: Managing 
the Transition to the Knowledge Media (Inglis, Eing and Joosten, 1999). In the original booklet, 
the Framework was presented in isolation and referred to simply as a “Quality Framework.” In 
Delivering Digitally, however, further material was added explaining how to apply the 
Framework within an institutional setting and providing an example of an application of the 
Framework. Delivering Digitally was initially published in a hardback edition. Within a year, 
however, it was reprinted as a paperback edition, and a second paperback edition has 
subsequently been published (Inglis, Eing and Joosten, 2002). In the second edition, the chapter 
that presents the Framework has been considerably revised. The focus of this chapter has been 
shifted more toward quality improvement and the explanation of the use of the Framework was 
expanded and some changes were made to the Framework itself. For these reasons, it is 
recommended that this is the edition to which readers should now refer. 

A number of universities have displayed interest in applying the Framework to their quality 
processes, either by using the Framework directly or by adapting aspects of the Framework. The 
Framework informed the development of quality assurance and quality improvement policies and 
processes at RMIT University where it was originally developed. It is currently being used at the 
University of Tasmania and the University of Melbourne. A workshop session on the 
Framework’s use for quality improvement was presented at the October 2003 meeting of the 
Australasian Council of Open, Distance and E-Feaming. Internationally, it has been reported that 
the Framework has been used at the University of Fund (Wigforss and Badersten, 2000a, 2000b). 
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Locating Each Framework in Relation to the 
Three Types of Quality Processes 

From what has been said about the origins of the two frameworks, it can be appreciated that the 
purposes for which they were developed were somewhat different. The framework described in 
McKinnon, Walker, and Davis (2000) was principally intended for use in comparing institutions. 
The framework described in Inglis, Ling, and Joosten (2002) was intended more for use in 
managing quality processes within institutions. These differences are then reflected in the 
structures of the respective frameworks, and the ways in which it is suggested that they be used. 
More will be said about those differences below. It is important not to make too strong a point of 
the difference in orientation. McKinnon, Walker and Davis (2000) point out that one of the uses 
to which the Benchmarking Framework can be put is identifying aspects of a university’s 
operations that could merit improvement. It is equally possible to use the Quality Improvement 
Framework for the purpose of comparing the performance of different types of institutions. Using 
either of the frameworks for quality assurance will require the setting of minimum performance 
standards, but that is obviously also a possibility. The difference in orientation is therefore one of 
relative alignment with the three quality processes, and the differences in alignment stem from 
the origins of the respective frameworks. The Benchmarking Framework was devised for the 
purpose of enabling institutions to compare themselves with other institutions with which they are 
competing for students, funding, and staff. The Quality Framework was devised for the purpose 
of enabling staff within institutions to institutionalize a quality improvement ethos. 


Comparing the Frameworks 

Frameworks of the type described here can be compared on the basis of a variety of factors. The 
purpose of this comparison is to provide some assistance in making choice decisions and 
therefore the frameworks will be compared on the basis of four factors, reflecting the major 
points of difference that are likely to affect such decisions: 

• Scope (the areas of application that they cover) 

• Type(s) of institution to which they are intended to be applied 

• Structures of the Frameworks 

• The ways in which they are meant to be applied 


Scope of the Frameworks 

One of the most immediately obvious differences between the two frameworks relates to their 
scope. The Benchmarking Framework was intended to enable universities to make comparisons 
across the full range of institutional functions. As well as encompassing aspects of teaching and 
learning, this framework therefore also deals with other aspects of a university’s role — notably 
research and community engagement. The Quality Improvement Framework, on the other hand, 
was specifically developed to contribute to the delivery of education and training programs that 
rely on the use of new learning technologies. This framework is therefore centered on those 
aspects of an educational institution’s operations that are concerned with teaching, program 
delivery, and student support. 
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In comparing the frameworks in relation to their scope, one finds areas of commonality as well as 
areas of marked difference. The Benchmarking Framework encompasses the area of teaching and 
learning that the Quality Improvement Framework has as its main focus. However, it covers a lot 
more besides. For staff specifically involved with teaching and learning, both frameworks 
therefore have something to offer. However, areas of overlap that lie outside those that are 
immediately obvious should not be overlooked. For example, although the Quality Improvement 
Framework focuses specifically on education and training delivery, it also deals with functions 
such as information technology infrastructure support, student learning support, and even 
institutional leadership. The Benchmarking Framework, for its part, has included benchmarks for 
Finance and Physical Infrastructure that overlap with Principle 1 of the Quality Improvement 
Framework dealing with the planning and management of resources and benchmarks for student 
support that overlap with Principle 10 of the Quality Improvement Framework dealing with the 
needs of learners. 

Institution type 

While the Benchmarking Framework is broader in scope than the Quality Improvement 
Framework, the Benchmarking Framework is designed for application to a much narrower range 
of institutions. The Benchmarking Framework is intended specifically for application to 
universities, and this is strongly reflected in the elements of the framework. As the authors point 
out, universities vary widely in terms of their missions, size, and focus but nevertheless have 
many features in common that set them apart from other types of educational institutions 
(McKinnon, Walker and Davis, 2000, p. 1). The Benchmarking Framework takes into account 
these characteristics of universities. The Quality Improvement Framework was devised for use in 
higher education but as it is presented in Inglis, Ling, and Joosten (2002) may be applied to any 
organization delivering education and/ or training programs relying on the new learning 
technologies. It could also be applied to vocational education and training providers, professional 
associations providing courses using new learning technologies, and commercial training centers. 
Where one can expect to find commonality between the frameworks therefore is in their 
application to universities. 


The structures of the Frameworks 

The most important differences between the frameworks relate to the ways in which the elements 
are conceptualized. These differences have, in turn, an important bearing on the ways in which 
the Frameworks are used as well as on the purposes for which they are used. 

Structure of the Benchmarking Framework. The Benchmarking Framework comprises a series of 
67 benchmarks, of which 25 have been designated as a core subset. As the Manual explains, the 
purpose of defining the core set was to provide a more manageable set of indicators to monitor on 
a regular basis, and the Manual goes into considerable detail in explaining the rationale for 
selection of the core subset and the implications of the benchmarks selected. 

The complete set of benchmarks covers the range of a university’s operations. They are grouped 
into nine areas: 

1 . Governance, Planning and Management 


2. External Impact 
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3. Finance and Physical Infrastructure 

4. Learning and Teaching 

5. Student Support 

6. Research 

7. Library and Information Services 

8. Internationalization 

9. Staff 

There are a number of aspects of the Benchmarking Framework that make the structure of the 
Framework difficult to grasp initially. First, the numbering of the individual benchmarks is 
according to the chapters of the Manual in which the indicators are placed. As a consequence, the 
numbering starts at 3.1 rather than 1.1. Secondly, the definitions of the individual benchmarks 
contain several components, but the purpose of these components and the ways in which it is 
intended that these should be used is not fully described. In most cases, the purpose and use are 
fairly obvious. This is not always the case, however. For example, the Levels component provides 
a criterion by which to judge an institution's practices but the status and origins of the criterion 
are not explained. 

Structure of the Quality Improvement Framework. The Quality Improvement Framework is 
organized around 10 key principles. These principles are meant to encompass the range of 
functions involved in supporting online delivery: 

1 . Informed planning and management of resources 

2. Sustained committed leadership 

3. Improving access for all clients, incorporating equity, and promoting cultural diversity 

4. Understanding the requirements of the learner and reflecting stakeholder requirements 

5. Design, development, and implementation of programs for effective and active learning 

6. Creating confident and committed staff with new competencies 

7. Managing and maintaining the technical infrastructure 

8. Evaluating for continuous improvement 

9. Provision of effective and efficient administrative services 

10. Supporting the needs of learners 

Associated with each of the principles is a set of best practice indicators. The indicators provide 
criteria for demonstrating whether the principles are being applied. 
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Application of the Framework also involves the development of checklists and evidence guides 
that are then the tools that are used in the field. The Quality Improvement Framework can 
therefore he thought of as a conceptual structure for guiding quality processes rather than as an 
evaluative instrument. 


Applying the frameworks 

The way in which it is anticipated that each of the frameworks will he used is reflected in their 
respective structures, as well as in the types of advice and assistance that are provided to users. 

Using the Benchmarking Framework involves contextualizing it to the situation to which it is 
being applied hy selecting the set of benchmarks that will be used. For each benchmark, a number 
of elements are defined: the area of institutional operations to which the benchmark applies, the 
type of element (that is, lagging, leading, learning), the rationale for the benchmark, good 
practice, and the levels of performance. 

This use of the Benchmarking Framework is assisted by the provision of graduated indicators that 
enable an institution to evaluate - in a rather basic fashion, it must be admitted - how well an 
institution scores against a particular benchmark. 

The Manual provides only a modest amount of guidance on the ways in which the Framework 
should be used. As has already been noted, the three general areas of application are pointed out. 
However, the procedures that should be followed when the Framework is used in any of these 
ways are not described. It is therefore left very much up to the user. This appears to be a 
significant shortcoming, given that any significant variation on the way that the Framework is 
used from institution to institution could greatly affect the validity of any comparisons that might 
be drawn. 

The Quality Improvement Framework also needs to be contextualized to the institutional 
situation, but this is accomplished somewhat differently. The Framework must be contextualized 
in two ways: firstly, in relation to the functions for which the particular organizational unit is 
responsible; and secondly, in relation to current institutional priorities. Because the Framework 
has been designed to be all-encompassing in the sense of identifying the range of functions that 
need to be supported by an education or training provider, whereas organizational units are 
typically responsible for only a subset of functions, each organizational unit needs to identify the 
principles and best practice indicators that are applicable to its role. Also, because most 
organizational units do not have the resources to work across all fronts at once, the Framework 
also needs to be contextualized to the priorities. This will involve identifying those principles and 
indicators that are the focus of attention for the immediate future. The implication of this second 
step is that the Framework will need to be recontextualized at regular intervals. 

The instrument that is used in the field is therefore not the Quality Improvement Framework 
itself, but the checklists that are derived from the Framework. As institutional priorities will 
change from year to year, these checklists will likewise change over time. 


Choosing between the Frameworks 

The purpose of erecting a conceptual structure is to guide thinking within a particular field of 
practice, which, in turn, may then guide action. It is appropriate, therefore, to consider the ways in 
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which each of the Frameworks is able to guide the thinking of teaching and administration about 
the practice of ODL. 

The type of question that a head of a center of ODL is likely to want to ask in relation to the 
possible application of the frameworks to a particular situation is: “Which Framework should I be 
using and how should I be using it?” 

From what has been said, it can be seen that the main differences between the frameworks lie in 
their scope, structures, and domains of application rather than in their intended functions. Both 
frameworks are capable of performing more or less the same functions within universities in 
relation to ODL. However, the ways in which the frameworks are designed to be used makes 
them more suited to some purposes than to others. When comparing the frameworks for the 
purpose of choosing between them it is therefore more profitable to begin by asking, “For what 
purposes is the framework being chosen?” than “Which framework best suits a particular 
function?” 

If the principal aim is to compare one’s own institution with others, then, depending on the level 
at which the comparison is being made, either framework could suffice. However, the 
Benchmarking Framework will support a more general comparison, while the Quality 
Improvement Framework will allow the comparison to be made at an operational level. On the 
other hand, if the principal aim is to provide a means of managing quality processes, then the 
Quality Improvement Framework will provide a more flexible and adaptable vehicle for 
achieving this aim. 

The Benchmarking Framework has benefited from the extensive consultation with senior 
university administrators, both in Australia and overseas, that went into the construction of its 
benchmarks. However, the same characteristics that recommend it for making comparisons 
among institutions at the same time diminish its value for managing quality improvement 
processes within institutions at the operational level. What it gains in general application, it loses 
in adaptability to the needs of operational units. The Quality Improvement Framework, on the 
other hand, is designed to adapt to the characteristics of institutions at all levels. Because the 
checklists and evidence guides are highly variable, the Quality Improvement Framework is able 
to be adapted to a wide range of institutional types and organizational structures. It can also easily 
take into account the stage of development of an organization. The more detailed explanation of 
the ways in which it can be used may be another factor that attracts those who have not had 
previous experience of introducing quality processes within educational institutions. 


Maintaining the currency of the Frameworks 

An inherent aspect of the concept of best practice is that what constitutes best practice will 
change over time. Maintaining a commitment to best practice entails looking for opportunities 
and avenues for improvement and the effect of the drive for improvement will be to raise the 
benchmarks over time. This is particularly likely to be the case in the area of flexible and e- 
learning, where changes in technology will offer opportunities to raise the quality of delivery and 
support services while at the same time lowering costs. 

The Quality Improvement Framework was developed in the context that existed at the time. It is 
possible that the Framework may not be able to accommodate fully developments that take place 
in what practitioners in ODL regard as best practice. No criteria exist at the present time by which 
to judge how long a framework of this type might remain current. However, given the level of 
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abstraction at which the framework has been specified, it is to be expected that the framework 
will date much more slowly than the practices to which it refers. 

The currency of the Benchmarking Framework will also be subject to change over time. It has 
already been pointed out that the originators of the Framework have themselves foreshadowed the 
need for further development. Indeed, the function of benchmarking, which, as explained earlier, 
draws a comparison with the best of current practice, involves adjusting to the benchmarks to take 
into account improvements in practice. A recent study, however, that began with the objectives of 
adding new elements covering student admission practices and student complaints procedures to 
the benchmarking framework and of reviewing benchmarking generally among universities found 
that the manual developed by McKinnon, Walker, and Davis (2002) was not seen by universities 
as a useful benchmarking tool and concluded that the manual “does not offer a solution to 
benchmarking in an increasingly ‘involve me’ evaluation world” (Gar lick and Pryor, 2004, p. 
viii). The findings of this study underscore the importance of one of the recurring themes in the 
literature relating to the application of quality processes: that systems and frameworks must be 
not only conceptually sound but also acceptable to stakeholders in the process. 

The Dark Side of the Introduction of Quality Processes 

There has been an implication in what has been said that the results produced by the application 
of quality processes are invariably desirable. In the ideal world, that may be so, but in the real 
world we usually fall considerably short of that for which we aim. In falling short of the ideal, we 
run the risk that the consequences of the intervention may be more problematic than the issues 
that the intervention is designed to address. This, in addition to the fact that the system needed to 
implement the quality processes itself, carries a cost. This cost is reflected particularly in the time 
of staff involved in the processes. 

One of the adverse consequences of the application of quality frameworks can be that attention is 
focused on certain aspects of programs (those identified in the framework), to the neglect of other 
aspects that may nevertheless be regarded as being of importance. To the extent that this happens, 
it could be considered a reflection of a deficiency in the design of a framework. The difficulty of 
achieving an ideal design, however, must nevertheless be recognized. 

An issue of greater concern is that the introduction of the use of quality frameworks may in 
certain circumstances have a stultifying effect on innovation. The corollary of increasing the 
value placed on certain specified activities is that the value placed on other activities that are not 
identified will be diminished. The way in which the use of quality frameworks has been applied is 
therefore likely to have a significant impact on the willingness of staff to break with existing 
conventions. 

Finally, as Garlick and Pryor (2004) point out, the top-down imposition (see also Cummings, 
Phillips, Tilbrook and Lowe, this issue) of indicators on an institution is likely to lead to staff 
members responding to the letter rather than to the intent of the process, thus undermining the 
purpose of the activity. 


Conclusion 

Both the Benchmarking and the Quality Improvement Frameworks represent potentially useful 
tools for managing quality processes. From this comparison, however, it is apparent that the 
applications for which each is suited are somewhat different. The suitability of the frameworks 
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for different purposes lies not just in the structures of the frameworks, hut also in their degree of 
abstraction, the ways in which they are intended to he used, and their adaptability to different 
contexts. 

For comparing the overall performance of universities in the area of ODL, the Benchmarking 
Framework has some applications. Flowever, when it comes to making operational decisions in 
relation to the management of individual organizational units, the Quality Improvement 
Framework is likely to be found more useful. The reasons for this lie not just in the fact that this 
was the purpose for which the framework was primarily developed, but also because it is more 
adaptable to individual contexts. 
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